Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pump/: Fix block at wait for MaxCommitTS when offline #701

Merged
merged 9 commits into from
Aug 7, 2019

Conversation

july2993
Copy link
Contributor

@july2993 july2993 commented Aug 5, 2019

What problem does this PR solve?

when offline:
1, ApplyAction change statue to be closing, reject write but accept fake
binlog
2, commitStatus() -> waitSafeToOffline(), we will write a fake binlog
and wait when storage.MaxCommitTS() >= commit ts of the fake binlog
...

after 1, may having some unwritten c-binlog(we will reject the write
request from tidb), and in sorter, block at s.cond.Wait case because no
more any item push.

What is changed and how it works?

only reject p-binlog when not online, and signal to wait up in sorter period.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    offline pump when write qps = 10k

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

when offline:
1, ApplyAction change statue to be closing, reject write but accept fake
binlog
2, commitStatus() -> waitSafeToOffline(), we will write a fake binlog
and wait when storage.MaxCommitTS() >= commit ts of the fake binlog
...

after 1, may having some unwritten c-binlog(we will reject the write
request from tidb), and in sorter, block at s.cond.Wait case because no
more any item push.
@july2993
Copy link
Contributor Author

july2993 commented Aug 5, 2019

/run-all-tests

pump/server.go Show resolved Hide resolved
@july2993
Copy link
Contributor Author

july2993 commented Aug 6, 2019

/run-all-tests

Copy link
Contributor

@zier-one zier-one left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pump/server.go Outdated Show resolved Hide resolved
pump/storage/sorter.go Show resolved Hide resolved
pump/storage/sorter.go Outdated Show resolved Hide resolved
@july2993
Copy link
Contributor Author

july2993 commented Aug 7, 2019

/run-all-tests

@july2993
Copy link
Contributor Author

july2993 commented Aug 7, 2019

/run-all-tests

Copy link
Contributor

@suzaku suzaku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@july2993 july2993 merged commit cc246ab into pingcap:master Aug 7, 2019
@july2993 july2993 deleted the hjh/offline branch August 7, 2019 08:49
july2993 added a commit to july2993/tidb-binlog that referenced this pull request Aug 7, 2019
* pump/: Fix block at wait for MaxCommitTS when offline

when offline:
1, ApplyAction change statue to be closing, reject write but accept fake
binlog
2, commitStatus() -> waitSafeToOffline(), we will write a fake binlog
and wait when storage.MaxCommitTS() >= commit ts of the fake binlog
...

after 1, may having some unwritten c-binlog(we will reject the write
request from tidb), and in sorter, block at s.cond.Wait case because no
more any item push.
IANTHEREAL pushed a commit that referenced this pull request Aug 8, 2019
* pump/: Fix block at wait for MaxCommitTS when offline

when offline:
1, ApplyAction change statue to be closing, reject write but accept fake
binlog
2, commitStatus() -> waitSafeToOffline(), we will write a fake binlog
and wait when storage.MaxCommitTS() >= commit ts of the fake binlog
...

after 1, may having some unwritten c-binlog(we will reject the write
request from tidb), and in sorter, block at s.cond.Wait case because no
more any item push.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants